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Background / Context: 

Description of prior research and its intellectual context. 

The evidence base for intervening with adolescent-aged readers encompasses a number 
of small-scale, investigator-led studies (Kamil, Bonnan, Dole, Krai, Salinger, & Torgesen, 2008; 
Roberts, Torgesen, Boardman, & Scammacca, 2008). Findings indicate that older students who 
struggle with word-level skills benefit from systematic instruction in decoding (Archer & 
Gleason, 2003; Scammacca, et al, 2007) and multi-syllabic words (Carlisle & Stone, 2005; 
Nagy, Berninger & Abbott, 2006) prior to or along with instruction in vocabulary (Baumann, 
Kame’enui, & Ash 2003) and comprehension strategies (Edmonds et al, 2009; Klingner, 
Vaughn, & Boardman, 2007). As word-level fluency improves, greater effort can be devoted to 
understanding what is read because fewer resources are necessary for the mechanics of 
identifying words (Kuhn & Stahl, 2000; National Institute of Child Health and Human 
Development, 2000; Reutzel & Hollingsworth, 1993), creating opportunities for improved 
comprehension. Older readers who are fluent, but nonetheless struggle with comprehension, may 
benefit from strategy instruction on monitoring, summarization, question generation, and related 
strategies, although strategy-related effects may be conditional on more substantive cognitive 
structures (Willingham, 2007) or on students’ developmental status (Cantrell, Alrnasi, Carter, 
Rintamaa, & Madden, 2010). Students who read with adequate fluency tend to read more words 
overall (Torgesen, 2005), are more motivated to read for information, and have increasingly 
greater access to the conceptual and background knowledge necessary for processing of complex 
text (Stanovich, 1986), compared to less fluent readers. Studies comprising the above research 
focus largely on the effectiveness of one or two instructional components or strategies. 

Large-scale, randomized trials featuring the above practices have been less encouraging. 
For example, Chamberlain, Daniels, Madden, and Slavin (2007) found no statistically significant 
differences on the Gates-MacGinitie comprehension subtest (ES = 0.1 1) after a year-long 
randomized implementation of Reading Edge, a Success For ^//-aligned comprehensive school- 
wide literacy model developed for use in 6 th grade. There were no differences (ES = -0.09) in the 
posttest reading comprehension for students (4 th through 6 th grade) participating in an intensive, 
research-based after-school intervention program (Kim, Samson, Fitzgerald, & Hartry, 2010). 
Vaughn and colleagues (Vaughn, Wanzek, Wexler, Barth, Cirino, Fletcher, Romain, Denton, 
Roberts, & Francis, 2010) reported no effects for a yearlong trial of a 50-minute daily 
intervention provided to at-risk middle school students. An evaluation of Enhanced Reading 
Opportunities found no statistically significant differences between two treatment groups and a 
business as usual condition on oral language vocabulary-related outcomes and very small 
differences on a measure of reading comprehension (ES = 0.09) (Somers, Corrin, Sepanik, T, 
Levin, & Zmach, 2010). Lang et al. (2009) provided 90-minutes of daily intensive reading 
interventions to older struggling readers over a 9-month period and found that low performing 
readers made no statistically significant gains in reading comprehension. Denton, Wexler, 
Vaughn, and Bryan (2008) reported similar results in a sample of middle school students 
provided with daily intensive intervention over the course of a school year (ES = 0. 10). 

A confluence of design- and program-related factors may account for differences in these 
two general groups of findings, and possibilities in this respect are increasingly prevalent in 
scholarly sources. The project ( R324A100022) summarized in this paper contributes to this 
ongoing discussion by investigating the impact (i.e., randomized design) of a comprehensive, 
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responsive intervention that combines a multi-component reading program with a school 
engagement, dropout prevention initiative for at-risk and struggling high school students. We 
summarize selected findings from year 1 . 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

The study evaluates the efficacy of an intensive, reading intervention, a dropout prevention 
intervention, and an intensive, reading intervention plus dropout prevention on high school 
students’ reading achievement and rates of dropout/school engagement. This paper focuses on 
the reading intervention and on reading outcomes. Data on the drop out intervention continues 
for the next 2 years. Our research questions include: 

(1) What is the efficacy of an intensive reading intervention, with adolescent struggling 
readers compared with a well-documented, school-implemented comparison group on posttest 
reading performance? 

(2) What is the moderating effect, if any, of primary language status and special education 
status? 

Setting: 

Description of the research location. 

Three diverse high schools in a large urban southwestern US district participated in the 
study, with approximately a third of the sample from each site. In the sampled schools, 
approximately 43.11% of students are Hispanic, 25.5 1% are White, 19.44% are African 
American, 7.85% are Asian, and 4.06% are Native American or biracial. Additionally, 42.6% of 
students in participating schools are economically disadvantaged. The schools are rated as 
Academically Acceptable for the 2010-2011 school year. 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 

Demographic details for sampled students (n = 375) are reported in Table 1. The majority 
of students are male (60.8%, n = 228) and Hispanic (44.3%, n = 166). An additional 17.6% (n 
=66) are Anglo, 33.1% (n =124) are African American, and 5.1% (n =19) are Asian. Of the 375 
students, 66 (17.6%) receive special education services (34 in the treatment and 32 in the control 
condition) and 70 are English Language Learners (30 in the treatment and 40 in the control 
condition). 

Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 

Description of reading intervention: 

Students in the reading conditions were provided a daily, 50 minute reading intervention 
class (Reading Interventions for Adolescents [RIA]) in groups of approximately 10 students that 
took the place of an elective. Students in the comparison and DO only conditions participated in 
an elective class such as art, music, athletics, or a foreign language. REWARDS (Archer, 
Gleason, & Vachon, 2003) review lessons provided the content for advanced word study. To 
address comprehension, we developed a series of 8-day instructional units aligned with 
recommendations from the recently published IES Adolescent Literacy Practice Guide (Kamil, et 
al., 2008) and implemented according to the principles of Collaborative Strategic Reading (CSR; 
Klingner, Vaughn, Dimino, Schumm & Bryant, 2001). CSR is a reciprocal teaching model where 
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students learn the principles of effective thinking (about text in this case) by participating, 
initially, with a group as the teacher explicitly “thinks aloud” about a given passage. Over time, 
individual students take over the “think aloud” role, and the teacher plays an increasingly 
consultative role. Textbooks from students’ English language arts, social studies, and science 
classes provided expository passages used in the 8-day units. Curriculum-based measures 
assessed content acquisition and vocabulary. The 8-day units were developed for instructional 
groups (versus individuals) and can be conceptualized as a “standard protocol.” However, their 
delivery in the CSR-affiliated model required a more responsive instructional stance, where 
teachers and students actively and transparently collaborated around the processing, 
comprehending, and using of text according to the strengths and learning needs of the students 
comprising each group. 

Research Design: 

Description of the research design. 

We conducted a randomized field trial to compare the effects of an intensive reading 
intervention with secondary students. Students who qualified for the study (N= 457) were 
randomly assigned to 1 of 4 conditions: (1) reading intervention only (R only), (2) dropout 
prevention intervention only (DO only), (3) reading plus dropout prevention intervention 
(R+DO), (4) a business as usual comparison condition (BAU). After assent and attrition, in the 
Fall of 2010 prior the intervention started, we had 86 students enrolled in the R only condition, 
104 students enrolled in the DO only condition, 84 students enrolled in the R+DO condition, and 
101 students enrolled in the BAU condition. Intervention was provided to students during their 
9th grade year (2010-2011) and will continue during their 10th (20110-2012) grade year. This 
paper reports the effects of the first year of reading intervention. 

Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

Data Collection: Students in the bottom 25 th percentile on the prior-year’s state high- 
stakes test were eligible for participation. Pretest occurred within the first month of school when 
intervention began and posttest during the last month of school. Trained project staff collected 
data. Researchers did not disclose treatment condition to test administrators. The reading 
achievement battery included the Test of Word Reading Efficiency (TOWRE; Torgesen, 

Wagner, & Rashotte, 1999), AIMSweb Maze (AIMSweb Maze-CBM, 2009), and the Gates- 
MacGinitie Reading Test (Gates & MacGinitie, 2000). 

Data Analysis: We compared students in the reading treatment groups (R only group or 
to the R+DO group) to the students in the reading comparison groups (DO only group or to the 
business-as-usual control group). The multilevel effects of reading intervention on students’ 
reading performance were estimated using multilevel modeling in MLwIN 2.23 (Rasbash, Steele, 
Browne, & Goldstein, 2009). We initially fit three-level models fit for each of the reading 
outcome variable to adjust for the clustering of students within classes and within school. Given 
the very small between-classes variance (between 0 - 5.6% of the total variance), two-level 
models, with students nested in classes, were subsequently fit. Pretest scores centered on the 
grand-mean as level- 1 covariates (Enders & Tofighi, 2007). The effect of treatment was modeled 
as a level-2 co variate. We calculated effect size as the ratio of the model-derived coefficient for 
treatment to the unadjusted pooled within-group standard deviation across conditions at posttest 
(i.e., Hedges’ g). 
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Findings / Results: 

Description of the main findings with specific details. 

Treatment effects (Research Question 1). Table 2 presents means and standard 
deviations for each group at pretest and posttest. Fixed and random effects for the random 
intercept model with Level 1 covariates are presented in Table 3. Students in treatment group 
(see Table 3) outperfonned students in the control group on the Gates-MacGinitie 
Comprehension subtest (y 0 i = 3.05, SE = 0.94, t = 3.24,/? < .05). This is equivalent to an effect 
size of .26 (Hedges’ g). Results were similar for the TOWRE Phonemic Decoding subtest, 
although the treatment effect estimate was somewhat smaller (yoi ~ 1-60, SE = 0.79, t = 2.02, p < 
.05; Hedges’ g = .12). Differences on the remaining reading measures (including Gates- 
MacGinitie Vocabulary subtest, TOWRE Sight Word subtest, and AIMSweb Maze) were not 
statistically significant. 

Student-level differences in treatment effect (Research Question 2). Treatment effects 
did not differ for English Language Learners (ELL) or for students with special needs status (see 
Table 3). The main effect of special education status was significant for Aims Web (703 = -3.37, 
SE= 1.72, t = -1.96,/? < .05) and TOWRE Sight Word (y 03 = -2.52, SE = 1.24, t = -2.03,/? < .05) 
indicating that students with special needs scored significantly lower on these two tests than non- 
special needs children regardless of treatment condition. 

Attrition bias. To determine whether attrition threatens the internal validity of the study 
we compared the characteristics of those remaining in the study across different treatment 
conditions (Cook &Campbell, 1979). Findings indicated no condition x attrition status 
interaction for any of the primary variables (Gates-MacGinitie, TOWRE, AIMSweb), suggesting 
that study attriters did not differ significantly at time 1 across conditions. 

Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

We summarize selected findings from year 1 of an ongoing IES -funded study on reading 
intervention and dropout prevention, with a focus on main effects of an integrated, data-driven 
model of reading instruction with older struggling readers, 9 th graders in this case. Earlier 
investigator-led research with struggling older readers has focused on discrete components of 
effective instruction. Larger, randomized studies in similar populations of students have 
generally taken a more programmatic approach (i.e., multi-component), though these programs 
tend to constrain opportunities to respond to students’ instructional needs. Because larger-scale 
implementations tend to result in lower levels of treatment fidelity (and higher levels of 
“contamination” of the counterfactual), programs in such settings often become routinized as a 
means of promoting fidelity (effectively so). However, the loss of responsiveness that 
accompanies increased standardization may diminish treatment effects for at-risk students. We 
test the effect of a responsive reading intervention in a relatively large-scale randomized trial. 
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Appendix B. Tables and Figures 
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Table 1 



Student demographics 



Characteristics 


Overall 
( n = 375) 


Treatment 
( n = 170) 


Control 
( n = 205) 




n 


% 


n 


% 


n 


% 


Gender 


Male 


228 


60.8 


105 


61.8 


123 


60 


Female 


147 


39.2 


65 


38.2 


82 


40 


Ethnicity 


Anglo 


66 


17.6 


32 


18.8 


34 


16.6 


African American 


124 


33.1 


58 


34.1 


66 


32.2 


Hispanic 


166 


44.3 


74 


43.5 


92 


44.9 


Asian 


19 


5.1 


6 


3.5 


13 


6.3 


English Language Learners 


70 


18.7 


30 


17.6 


40 


19.6 


Special education 


66 


17.6 


34 


20 


32 


15.7 
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Table 2 



Means and standard deviations for reading measures across pretest and posttest 



Reading measures 


Pretest 


Posttest 


M 


SD 


M 


SD 


Gates Comprehension 










Treatment 


88.6 


12.6 


89.5 


11.8 


Control 


91.7 


11.1 


88.7 


11.8 


Gates Vocabulary 










Treatment 


89.1 


9.47 


88.0 


12.2 


Control 


89.8 


9.14 


89.4 


12.3 


TOWRE Phonemic Decoding 










Treatment 


88.9 


11.4 


93.2 


13.1 


Control 


91.1 


12.6 


94.5 


13.8 


TOWRE Sight word 










Treatment 


88.0 


9.62 


91.1 


10.4 


Control 


89.5 


9.75 


91.9 


10.0 


AIMS Maze 










Treatment 


96.8 


12.1 


101 


12.1 


Control 


100.0 


11.3 


102 


11.8 
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Table 3 



Multilevel model of fixed and random effects for reading outcomes 





Fixed Effects 




Random effects 








Students within school 


Students 




Predictor 


Estimate 






Gates Comprehension 


Intercept 


87.41 (.63) a 


.00 (.00) 


72.67 (5.57) 




Pretest 


0.69 (.04) 








Treatment b 


3.05* (0.94) 






Gates Vocabulary 


Intercept 


88.91 (.64) 


.06 (.58) 


73.57 (5.65) 




Pretest 


0.95 (.05) 








Treatment 


-0.47 (0.93) 






TOWRE Sight Word 


Intercept 


91.36 (.45) 


.00 (.00) 


36.95 (2.84) 




Pretest 


0.81 (.03) 








Treatment 


0.63 (.67) 






TOWRE Phonemic Decoding Intercept 


93.33 (.57) 


.15 (.50) 


50.56(3.93) 




Pretest 


0.96 (.03) 








Treatment 


1.60* (.79) 






AIMS 


Intercept 


101.75 (1.56) 


6.09 (5.49) 


72.52 (5.57) 




Pretest 


0.69 (.04) 






-KT , a Oi 1 1 


Treatment 

n BTT" 


0.02 (0.93) 







Note. a Standard errors are in parentheses; b Reference group is control; * < .05. 
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